Control of variable-focus lenses in a mixed-reality device for presbyopes

ABSTRACT

Variable-focus lenses are arranged as a lens pair that work on opposite sides of a see-through optical combiner used in a mixed-reality head-mounted display (HMD) device. An eye-side variable-focus lens is configured as a negative lens over an eyebox of the see-through optical combiner to enable virtual-world objects to be set at a close distance. The negative lens is compensated by its conjugate using a real-world-side variable-focus lens configured as a positive lens to provide for an unperturbed see-through experience. For non-presbyopes, the powers of the lenses are perfectly offset. For presbyopes, the lens powers may be mismatched at times to provide simultaneous views of both virtual-world and real-world objects on the display in sharp focus. Responsively an eye tracker indicating that the user is engaged in close viewing, optical power is added to the real-world-side lens to push close real-world objects optically farther away and into sharp focus for the presbyopic user.

BACKGROUND

Presbyopia is an ocular condition in which one loses the ability tooptically focus (or accommodate) one’s eyes to varying distances. Whilethe age of people experiencing the onset of presbyopia (referred to as“presbyopes”) may vary, rapid reduction in accommodation range typicallybegins around 45 years of age, with virtually 100 percent of people overthe age of 55 years being presbyopic. For presbyopes who began withnormal vision, their natural accommodation state effectively rests atoptical infinity, making it hard to accommodate on near (e.g., < 1 m)objects.

SUMMARY

Variable-focus lenses are arranged as a conjugate lens pair that work onopposite sides of a see-through optical combiner used in a mixed-realityhead-mounted display (HMD) device in which virtual images aresuperimposed over views of real-world objects. An eye-sidevariable-focus lens is configured as a negative lens over an eyebox ofthe optical combiner to enable virtual images to be placed atpredetermined (i.e., non-infinite) depth from the device user to enhancevisual comfort. The negative lens is compensated by its conjugate usinga real-world-side variable-focus lens that is configured as a positivelens to provide for an unperturbed see-through experience.

For non-presbyopes (i.e., emmetropes), the powers of the negative andpositive lenses are perfectly offset so that no net optical power isprovided to the real world viewed through the see-through opticalcombiner. For a presbyopic HMD device user, the lens powers may bemismatched at times to enable the user to simultaneously view bothvirtual-world and real-world objects on the display in sharp focus.Responsively to an eye tracker in the HMD device that indicates that theuser is engaged in close viewing, optical power is added to thereal-world-side variable-focus lens to push close real-world objectsoptically farther away and into sharp focus for the presbyopic user.

In an illustrative embodiment, the variable-focus lens pair may beconfigured to work in combination to integrate a user’s corrective lensprescription into the HMD device. Such integration enables the HMDdevice to be utilized without the need for the user to wear glasses orcontact lenses. The HMD device can replicate dual-prescriptionfunctionality to correct for both near and far vision impairments of theuser by adapting the eye-side variable focus lens in a modifiedconfiguration to include the user’s corrective prescription for bothclose and far use cases. The real-world-side lens may provide additionaloptical power when the eye tracker indicates that the user is engaged inclose viewing to push close real-world objects optically farther awayand into sharp focus.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter. Furthermore, the claimed subject matter is not limited toimplementations that solve any or all disadvantages noted in any part ofthis disclosure.

DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a pictorial partially cutaway view of an illustrative HMDdevice that is configured with the present control of variable-focuslenses in a mixed-reality device for presbyopes;

FIG. 2 illustratively shows holographic virtual images that areoverlayed onto real-world images within a field of view (FOV) of amixed-reality head-mounted display (HMD) device;

FIGS. 3A, 3B, and 3C show illustrative partially spherical wavefrontsthat are respectively associated with a distant object, an object atinfinity, and a nearby object;

FIG. 4 shows an illustrative negative lens that provides for a virtualimage that is located at a focal point of the lens;

FIG. 5 shows a side view of an illustrative virtual display system thatincludes a waveguide-based optical combiner providing for rendering ofvirtual images in a focal plane having predetermined depth that may beused in an HMD device;

FIG. 6 shows a side view of an illustrative virtual display system inwhich variable-focus lenses are arranged as a conjugate lens pair;

FIG. 7 shows a side view of an illustrative virtual display system inoperative relationship with HMD device components including an eyetracking system, optical power controller, and processors;

FIG. 8 is a table that shows illustrative operational configurations forthe variable-focus lens pair for different user types and use cases;

FIG. 9 is a flowchart of an illustrative workflow for operating avariable-focus lens pair in an HMD device;

FIG. 10 is a flowchart of an illustrative method for operating anelectronic device that includes an eye tracker and a mixed-realitysee-through optical display system for showing scenes comprising virtualimages that are superimposed over views of real-world objects;

FIG. 11 shows a pictorial front view of an illustrative sealed visorthat may be used as a component of an HMD device;

FIG. 12 shows a pictorial rear view of an illustrative sealed visor;

FIG. 13 shows a partially disassembled view of an illustrative sealedvisor;

FIG. 14 shows an illustrative arrangement of diffractive opticalelements (DOEs) configured for in-coupling, exit pupil expansion in twodirections, and out-coupling;

FIG. 15 shows a side view of an illustrative assembly of threewaveguides with integrated coupling elements that are stacked to form anoptical combiner, in which each waveguide handles a different color inan RGB (red, green, blue) color model;

FIG. 16 is a pictorial view of an illustrative example of avirtual-reality or mixed-reality HMD device that may use the presentcontrol of variable-focus lenses in a mixed-reality device forpresbyopes;

FIG. 17 shows a block diagram of an illustrative example of avirtual-reality or mixed-reality HMD device that may use the presentcontrol of variable-focus lenses in a mixed-reality device forpresbyopes; and

FIG. 18 schematically shows an illustrative example of a computingsystem that can enact one or more of the methods and processes describedabove for the present control of variable-focus lenses in amixed-reality device for presbyopes.

Like reference numerals indicate like elements in the drawings. Elementsare not drawn to scale unless otherwise indicated.

DETAILED DESCRIPTION

Presbyopes (i.e., persons who have presbyopia) present a uniquechallenge to mixed-reality HMD devices that do not afford wearingspectacles during use. These devices usually have a fixed focal planefor the digital content they overlay on the real world. Thus, even if apresbyopic user did not require glasses for distance viewing and thevirtual reality images were placed at a far distance that appearedsharp, if the user needed to look at a close-by object (e.g., theirsmartphone), they would not be able to see that object sharply withoutremoving the HMD device and donning reading glasses. Meanwhile, ifdigital content were meant to be overlaid on nearby real-world objects,the user would not be able to see both the digital and real content insharp focus at the same time.

While some conventional HMD devices can provide comfortable userexperiences for users who wear glasses, such devices typically do notaccommodate presbyopes. The disclosed arrangement provides for controlof a pair of variable-focus lenses to enable presbyopes to sharply andcomfortably view both virtual-world and real-world objects at anydistance.

Turning now to the drawings, FIG. 1 shows a pictorial partially cutawayview of an illustrative mixed-reality HMD device 100 that is configuredto implement the present control of variable-focus lenses in amixed-reality device for presbyopes. In this example, the HMD deviceincludes a display device 105 and a frame 110 that wraps around the headof a user 115 to position the display device near the user’s eyes toprovide a mixed-reality experience to the user.

Any suitable technology and configuration may be used to display virtualimages, which may also be referred to as holograms or holographicimages, using the display device 105. For a mixed-reality experience,the display device may be see-through so that the user of the HMD device100 can view physical, real-world objects in the physical environmentover which pixels for virtual objects are overlayed. For example, thedisplay device may include one or more partially transparent waveguidesused in conjunction with a virtual image source such as, for example, amicrodisplay comprising RGB (red, green, blue) LEDs (light emittingdiodes), an organic LED (OLED) array, liquid crystal on silicon (LCoS)device, and/or MEMS device, or any other suitable displays ormicrodisplays operating in transmission, reflection, or emission. Thevirtual image source may also include electronics such as processors,optical components such as mirrors and/or lenses, and/or mechanical andother components that enable a virtual display to be composed andprovide one or more input optical beams to the display system. Virtualimage sources may be referred to as light or display engines in somecontexts.

In some implementations, outward facing cameras 120 that are configuredto capture images of the surrounding physical environment may beprovided. Such captured images may be rendered on the display device 105along with computer-generated virtual images that augment the capturedimages of the physical environment.

The frame 110 may further support additional components of the HMDdevice 100, including a processor 125, an inertial measurement unit(IMU) 130, and an eye tracker 135. In some implementations, the eyetracker can be configured to support one or more of vergence trackingand/or gaze tracking functions. The processor may include logic andassociated computer memory configured to receive sensory signals fromthe IMU and other sensors, to provide display signals to the displaydevice 105, to derive information from collected data, and to enactvarious control processes described herein.

The display device 105 may be arranged in some implementations as anear-eye display. In a near-eye display, the virtual image source doesnot actually shine the images on a surface such as a glass lens tocreate the display for the user. This is not feasible because the humaneye cannot focus on something that is that close. Rather than create avisible image on a surface, the near-eye display uses an optical systemto form a pupil and the user’s eye acts as the last element in theoptical chain and converts the light from the pupil into an image on theeye’s retina as a virtual display. It may be appreciated that the exitpupil is a virtual aperture in an optical system. Only rays which passthrough this virtual aperture can exit the system. Thus, the exit pupildescribes a minimum diameter of the holographic virtual image lightafter leaving the display system. The exit pupil defines the eyeboxwhich comprises a spatial range of eye positions of the user in whichthe holographic virtual images projected by the display device arevisible.

FIG. 2 shows the HMD device 100 worn by a user 115 as configured formixed-reality experiences in which the display device 105 is configuredas a near-eye display system having at least a partially transparent,see-through waveguide, among various other components, and may befurther adapted to utilize variable-focus lenses in accordance with theprinciples discussed herein. As noted above, a virtual image source (notshown) generates holographic virtual images that are guided by thewaveguide in the display device to the user. Being see-through, thewaveguide in the display device enables the user to perceive light fromthe real world.

The see-through waveguide-based display device 105 can renderholographic images of various virtual objects that are superimposed overthe real-world images that are collectively viewed using the see-throughwaveguide display to thereby create a mixed-reality environment 200within the HMD device’s FOV (field of view) 220. It is noted that theFOV of the real world and the FOV of the holographic images in thevirtual world are not necessarily identical, as the virtual FOV providedby the display device is typically a subset of the real FOV. FOV istypically described as an angular parameter in horizontal, vertical, ordiagonal dimensions.

It is noted that FOV is just one of many parameters that are typicallyconsidered and balanced by HMD device designers to meet the requirementsof a particular implementation. For example, such parameters may includeeyebox size, brightness, transparency and duty time, contrast,resolution, color fidelity, depth perception, size, weight, form factor,and user comfort (i.e., wearable, visual, and social), among others.

In the illustrative example shown in FIG. 2 , the user 115 is physicallywalking in a real-world urban area that includes city streets withvarious buildings, stores, etc., with a countryside in the distance. TheFOV of the cityscape viewed on HMD device 100 changes as the user movesthrough the real-world environment and the device can render staticand/or dynamic virtual images over the real-world view. In thisillustrative example, the holographic virtual images include a tag 225that identifies a restaurant business and directions 230 to a place ofinterest in the city. The mixed-reality environment 200 seen visually onthe waveguide-based display device may also be supplemented by audioand/or tactile/haptic sensations produced by the HMD device in someimplementations.

Virtual images and digital content can be located in various positionswithin the FOV along all three axes of the coordinate system 235. Theimmersiveness of the content in three dimensions may be enhanced as thereach of the display along the “z” axis extends from the near fieldfocus plane (i.e., generally within arm’s length of the HMD device user)to the far field focus plane (i.e., generally beyond arm’s reach) tofacilitate arm’s length virtual display interactions. Many mixed-realityHMD device experiences will employ a mix of near-field and far-fieldvisual components. The boundary between near and far fields is notnecessarily strictly defined and can vary by implementation. Forexample, distances beyond 2 m may be considered as a part of the farfield in some mixed-reality HMD device scenarios.

During natural viewing, the human visual system relies on multiplesources of information, or “cues,” to interpret three-dimensional shapesand the relative positions of objects. Some cues rely only on a singleeye (monocular cues), including linear perspective, familiar size,occlusion, depth-of-field blur, and accommodation. Other cues rely onboth eyes (binocular cues), and include vergence (essentially therelative rotations of the eyes required to look at an object) andbinocular disparity (the pattern of differences between the projectionsof the scene on the back of the two eyes).

To view objects clearly, humans must accommodate, or adjust their eyes’focus, to the distance of the object. At the same time, the rotation ofboth eyes must converge to the object’s distance to avoid seeing doubleimages. In natural viewing, vergence and accommodation are linked. Whenviewing something near (e.g., a housefly close to the nose) the eyescross and accommodate to a near point. Conversely, when viewingsomething at optical infinity, the eyes’ lines of sight become parallel,and the eyes’ lenses accommodate to infinity.

In typical HMD devices, users will always accommodate to the focaldistance of the display (to get a sharp image) but converge to thedistance of the object of interest (to get a single image). When usersaccommodate and converge to different distances, the natural linkbetween the two cues must be broken and this can lead to visualdiscomfort or fatigue due to such vergence-accommodation conflict (VAC).Accordingly, to maximize the quality of the user experience and comfortwith the HMD device 100, virtual images may be rendered in a plane toappear at a constant distance from the user’s eyes. For example, virtualimages, including the images 225 and 230, can be set at a fixed depth(e.g., 2 m) from the user 115. Thus, the user will always accommodatenear 2 m to maintain a clear image in the HMD device. It may beappreciated that 2 m is an illustrative distance and is intended to benon-limiting. Other distances may be utilized, and virtual images maytypically be optimally placed at distances between 1.5 and 5 m from theHMD device user for many applications of a mixed-reality HMD devicewhile ensuring user comfort, however in some applications and use cases,virtual images can be rendered more closely to the user.

In the real world as shown in FIG. 3A, light rays 305 from distantobjects 310 reaching an eye of a user 115 are almost parallel.Real-world objects at optical infinity (roughly around 6 m and fartherfor normal vision) have light rays 320 that are exactly parallel whenreaching the eye, as shown in FIG. 3B. Light rays 325 from a nearbyreal-world object 330 reach the eye with different, more divergentangles, as shown in FIG. 3C, compared to those for more distant objects.

Various approaches may be utilized to render virtual images with thesuitable divergent angles to thereby appear at the targeted depth offocus. For example, FIG. 4 shows that a negative (i.e., concave) lens405 can diverge the collimated/parallel rays 450 that are received froma conventional output coupler element (not shown) in an HMD device toproduce a holographic virtual image having a location that is apparentto the user at a focal point, F (as indicated by reference numeral 415),that is determined by the focal length of the lens. For example, invarious mixed-reality HMD device scenarios, focal lengths can rangebetween -0.2 to -3.0 diopters (i.e., 33 cm to 5 m) to position virtualobjects from the boundary of the far field (near infinity) to slightlymore than one foot away. As shown, the rays from the negative lensarriving at the user’s eye 115 are non-parallel and divergent andconverge using the eye’s internal lens to form the image on the retina,as indicated by reference numeral 420.

FIG. 5 shows a simplified side view of an illustrative virtual displaysystem 500 that is incorporated into the display device 105 (FIG. 1 )and which may be used in the HMD device 100 to render virtual images.The virtual display system may function as an optical combiner bysuperimposing the rendered virtual images over the user’s view of lightfrom real-world objects to thus form a mixed-reality display.

It is noted that the side view of FIG. 5 shows virtual displaycomponents for a single eye of the user 115. However, it may beappreciated that the components can be extended such that separatedisplays are provided for each eye of the user in binocularimplementations. Such arrangement may facilitate, for example,stereoscopic rendering of virtual images in the FOV of the HMD deviceand enable prescription lens integration, as discussed below, on aper-eye basis.

The display system includes at least one partially transparent (i.e.,see-through) waveguide 510 that is configured to propagate visiblelight. While a single waveguide is shown in FIG. 5 for the sake ofclarity in exposition of the present principles, it will be appreciatedthat a plurality of waveguides may be utilized in some applications. Forexample, a stack of two or three waveguides can support a red, green,blue (RGB) color model that is utilized for rendering full color virtualimages in some cases.

The waveguide 510 facilitates light transmission between the virtualimage source and the eye. One or more waveguides can be utilized in thenear-eye display system because they are transparent and because theyare generally small and lightweight. This is desirable in applicationssuch as HMD devices where size and weight are generally sought to beminimized for reasons of performance and user comfort. Use of thewaveguide 510 can enable the virtual image source to be located out ofthe way, for example on the side of the user’s head or near theforehead, leaving only a relatively small, light, and transparentwaveguide optical element in front of the eyes.

In an illustrative implementation, the waveguide 510 operates using aprinciple of total internal reflection (TIR) so that light can becoupled among the various optical elements in the HMD device 100. TIR isa phenomenon which occurs when a propagating light wave strikes a mediumboundary (e.g., as provided by the optical substrate of a waveguide) atan angle larger than the critical angle with respect to the normal tothe surface. In other words, the critical angle (θ_(c)) is the angle ofincidence above which TIR occurs, which is given by Snell’s Law, as isknown in the art. More specifically, Snell’s law states that thecritical angle (θ_(c)) is specified using the following equation:

θ_(c) = sin⁻¹(n2/n1)

where θ_(c) is the critical angle for two optical mediums (e.g., thewaveguide substrate and air or some other medium that is adjacent to thesubstrate) that meet at a medium boundary, n1 is the index of refractionof the optical medium in which light is traveling towards the mediumboundary (e.g., the waveguide substrate, once the light is coupledtherein), and n2 is the index of refraction of the optical medium beyondthe medium boundary (e.g., air or some other medium adjacent to thewaveguide substrate).

The user 115 can look through the waveguide 510 to see real-worldobjects on the real-world side of the display device 105 (the real-worldside is indicated by reference numeral 512 in FIG. 5 ). For the virtualpart of the FOV of the display system, virtual image light 515 isprovided by a virtual image source 520 (e.g., a microdisplay or lightengine, etc.). The virtual image light is in-coupled to the waveguide byan input coupler 525 and propagated through the waveguide in totalinternal reflection. The image light is out-coupled from the waveguideby an output coupler 530. The combination of see-through waveguide andcoupling elements may be referred to as a mixed-reality optical combiner535 because it functions to combine real-world and virtual-world imagesinto a single display.

Typically, in such waveguide-based optical combiners, the input pupilneeds to be formed over a collimated field, otherwise each waveguideexit pupil will produce an image at a slightly different distance. Thisresults in a mixed visual experience in which images are overlappingwith different focal depths in an optical phenomenon known as focusspread.

In some embodiments, the input coupler 525 and output coupler 530 may beconfigured as diffractive optical elements (DOEs). DOEs may comprise,for example, surface relief grating (SRG) structures and volumetricholographic grating (VHG) structures. An intermediate DOE (not shown)may also be disposed in the light path between the input coupler andoutput coupler in some cases. The intermediate DOE may be configured toprovide exit pupil expansion in one direction (e.g., horizontal) whilethe output coupler may be configured to provide exit pupil expansion ina second direction (e.g., vertical).

In alternative embodiments, the optical combiner functionality providedby the waveguide and DOEs may be implemented using a reflectivewaveguide combiner. For example, partially reflective surfaces may beembedded in a waveguide and/or stacked in a geometric array to implementan optical combiner that uses partial field propagation. The reflectorscan be half-tone, dielectric, holographic, polarized thin layer, or befractured into a Fresnel element.

In other embodiments, the principles of the present control ofvariable-focus lenses in a mixed-reality device for presbyopes may beimplemented using a reflective waveguide combiner with any suitablein-coupling and/or out-coupling methods. A reflective waveguide combinermay utilize a single waveguide in some implementations for all colors inthe virtual images which may be desirable in some applications. Bycomparison, diffractive combiners typically require multiple waveguidesto meet a target FOV in polychromatic applications due to limitations onangular range that are dictated by the waveguide TIR condition.

The present control of variable-focus lenses in a mixed-reality devicefor presbyopes may also be utilized with various otherwaveguide/coupling configurations beyond reflective and diffractive. Forexample, it may be appreciated that the principles of the presentinvention may be alternatively applied to waveguides that arerefractive, polarized, hybrid diffractive/refractive, phase multiplexedholographic, and/or achromatic metasurfaces.

A variable-focus lens 540 configured to function as a negative lens islocated on the eye side of the waveguide 510 (the eye side is indicatedby reference numeral 514 in FIG. 5 ). The negative lens acts over theentire extent of the eyebox associated with the user’s eye to therebycreate the diverging rays 545 from the collimated rays 550 that exit theoutput coupler 530. When the virtual image source 520 is operated toproject virtual images that are in-coupled into the waveguide 510, theoutput diverging rays present the virtual images at a predeterminedfocal depth, d, from the display system at an apparent or virtual pointof focus, F. For example, if the negative lens is configured with -0.5diopters of optical power, then d is equal to 2 m.

To ensure that the user’s view of the real world remains unperturbed bythe negative lens, a variable-focus lens 605 is configured to functionas a conjugate positive (i.e., convex) lens, as shown in FIG. 6 . Thisvariable-focus lens is located on the real-world side of the waveguide510 to compensate for the impact of the negative lens on the eye side.The conjugate pair of positive and negative lenses may be referred to asa push-pull lens pair in some contexts. For example, if the eye sidevariable-focus lens is controlled to provide -0.5 diopters of opticalpower, then the real-world side lens is controlled to provide anopposite +0.5 diopters of optical power to cancel out the effect of thenegative lens. Accordingly, light 610 reflected from a real-world object615 reaches the user with no net optical power being applied by thecombined operations of the pair of variable-focus lenses. In thisexample, the object is in the distance so the parallel rays ofreal-world light incident on the display system 500 remain parallel whenviewed by the user 115.

The eye-side variable-focus lens 540 and real-world-side variable-focuslens 605 may be implemented using various known technologies.Variable-focus lenses may also be referred to as “tunable” lenses.Exemplary technologies include liquid oil push/pull, liquid crystal,reflective MEMS (micro-electromechanical system), MEMS Fresnelstructures, geometric phase holograms, meta-surface optical elements,deformable membranes, Alvarez lenses, multi-order DOEs, combinationsthereof, and the like. The lenses may be implemented using singleoptical elements in some applications, or as arrays in otherapplications.

FIG. 7 is a side view the virtual display system 500 in operativerelationship with HMD device components including an eye tracker 705,optical power controller 710, and one or more processors 715. Thecomponents and the variable-focus lenses 540 and 605 are operativelycoupled by one or more buses as representatively indicated by referencenumeral 720. The components may be disposed in a frame (not shown) orother suitable structure of the HMD device 100 or the exemplary HMDdevice 1600 shown in FIGS. 16 and 17 and described in the accompanyingtext.

The eye tracker 705 is operatively coupled to one or more illuminationsources 725 and one or more sensors 730. For example, the illuminationsources may comprise IR (infrared) LEDs that are located around theperiphery of the display system 500 (FIG. 5 ) and/or optical combiner535 and/or be disposed in some other suitable HMD device component suchas a frame. The eye tracker illumination sources can function as glintsources and/or provide general or structured illumination of the user’seye features. The eye tracker sensors may comprise inward-facing camerasthat have sensitivity, for example, to IR light. Image-based and/orfeature-based eye tracking, or other suitable eye-tracking techniquesmay be utilized to meet requirements of an implementation of the presentcontrol of variable-focus lenses in a mixed-reality device forpresbyopes.

In an illustrative example, the IR light from the illumination sources725 cause highly visible reflections, and the eye tracker sensors 730capture an image of the eye showing these reflections. The imagescaptured by the sensors are used to identify the reflection of the lightsource on the cornea (i.e., “glints”) and in the pupil. Typically, avector formed by the angle between the cornea and pupil reflections maybe calculated using real-time image analysis, and the vector directioncombined with other geometrical features of the reflections is then usedto determine where the user is looking - the gaze point - and calculateeye movement, location, and orientation.

During operation of the HMD device 100, the optical power controller 710controllably varies the optical power of the eye-side variable-focuslens 540 and real-world-side variable focus lens 605. Different amountsof optical power may be utilized at the eye-side variable-focus lenswhen configured as a negative lens to provide for focal planes that arelocated at different fixed or variable distances to suit requirements ofa particular application. The power of the negative lens does not affectthe zeroth diffraction order that travels in TIR down the waveguide 510(i.e., from top to bottom in the drawings), but instead only thediffracted out-coupled field. In addition, the see-through field is notaffected by the negative lens because whatever portion of thesee-through field that is diffracted by the output coupler 530 istrapped by TIR in the waveguide and is therefore not transmitted to theuser’s eye.

A static lens 735 may be optionally utilized in some implementations ofthe HMD device 100. For example, the static lens may be implemented asan optical insert to a portion of the HMD device such as a sealed visorshown in FIGS. 11-13 and described in the accompanying text. In some HMDdevices having size and space limits due to eyebox and/or form factorconsiderations, it may not be comfortable or possible for users to wearprescription glasses. The static lens can be provided to correctimpairments in the vision of the user 115 and may comprise, for example,the user’s corrective lens prescription for glasses or contact lenses.The static lens may be used in combination with a modified configurationfor the eye-side variable-focus lens discussed below in some scenarios.Worldwide, visual impairments due to refractive errors are distributedamong people with myopia, hyperopia, and presbyopia. Corrections formost of the population fall between -6.0 and +4.0 diopters.

FIG. 8 provides an illustrative table 800 that shows illustrativeoperational configurations for the variable-focus lens pair fordifferent user types and use cases. It may be noted that FIG. 8 refersto the elements shown in FIG. 7 . Table 800 shows how the optical powercontroller 710 may controllably vary the optical power of eye-sidevariable-focus lens 540 and real-world-side variable-focus lens 605 fordifferent types of users and HMD device use cases. Two different typesof presbyopic users are shown in the first column 805 of the table. User1 is able to see far away real-world objects clearly without glasses.User 1 may have always had clear (i.e., emmetropic) vision but developedpresbyopia with age. User 1 may currently use reading glasses to seeclose real-world objects and read text. For example, prescriptions forreading glasses typically increase by 0.25 diopters, such as +1.00,+1.25, +1.50, and so on.

User 2 may have developed myopia as a child and is unable to see faraway real-world objects clearly without corrective lenses such asglasses or contacts. To deal with presbyopia, user 2 may currently usebifocals or progressive lenses, or wear contact lenses for distancevision and don reading glasses for close accommodation. Monovision isanother solution in which different accommodative distances are providedeach eye of the user via contact lenses or surgical methods, forexample. User 2 may also remove or lift their glasses to focus on nearobjects in some situations.

The second column 810 in table 800 shows two use cases for a presbyopicuser of an HMD device, including far viewing and close viewing. It isnoted that the terms “close” and “far” are relative to each other andthat specific distances associated with each term can vary by contextand application of the present principles. Close regions of interest aregenerally within an arm’s length of the user, for example < 1 m andwithin the near field of an HMD device. The far field for the device maygenerally start around 2 m and a user’s eye generally accommodates tooptical infinity around distances of 6 m.

As noted above, virtual images may typically be displayed at fixed focalplane depths of around 1.25 to 2.5 m in mixed-reality HMD and otherimmersive devices to reduce user discomfort due to VAC. Accordingly, intypical implementations, an objective of the optical power controller710 is to enable presbyopes to simultaneously view both close virtualand real-world objects in sharp focus through the HMD device.

The third column 815 in table 800 shows the operations of the eye-sidevariable-focus lens 540 responsive to the optical power controller 710for each use case and for each user type. The fourth column 820 showsthe operations of the real-world-side variable-focus lens 605 responsiveto the optical power controller for each use case and for each usertype.

For user 1 during far viewing, the eye-side variable-focus lens 540 isoperated in its baseline configuration to support the rendering ofvirtual images at some predetermined mixed-reality focal plane depth.For example, in an illustrative and non-limiting embodiment, the opticalpower controller 710 can set the optical power of the eye-sidevariable-focus lens at -0.5 diopters to fix the mixed-reality focalplane at 2 m. In alternative embodiments, focus tuning for the virtualimages at some non-infinite distance may be implemented in the opticaldisplay system before light for the virtual images is out-coupled fromthe waveguide to the user’s eye. In such alternative embodiments, it maybe appreciated that the out-coupled light is not necessarily collimated,and thus the optical power of the eye-side variable-focus lens may beset by the optical power controller to zero or some other suitable valuefor its baseline configuration. For example, with an optical combineremploying a reflective waveguide having no exit pupil replication, focustuning may be performed at the virtual image source, at a tunabledisplay engine, or using some other suitable technique. In anotheralternative embodiment, focus tuning of the virtual images may beperformed by the output-coupler.

The optical power controller 710 may operate the real-world-sidevariable-focus lens 605 in its baseline configuration in which theoptical power provided by the eye-side lens is canceled out forreal-world image light 610 entering the see-through HMD display system.Here, for example, the baseline configuration for the real-world-sidelens may be +0.5 diopters so that the net optical power applied by thelens pair to light from real-world objects equals zero.

For close viewing by user 1, the optical power controller 710 alsoconfigures the eye-side variable-focus lens 540 to support the renderingof virtual images at a predetermined mixed-reality focal plane depth,for example 2 m. In addition, the optical power controller 710 addsoptical power to the real-world-side variable-focus lens 605 to pushclose real-world objects optically farther away and into sharp focus foruser 1. The amount of added optical power can vary according to one ormore of degree of presbyopia experienced by user 1, the amount ofambient light, or other factors. For example, the added optical powercould be +1.5 diopters for moderate presbyopia correction.

For far viewing by user 2, the real-world-side variable-focus lens 605is operated by the optical power controller 710 in its baselineconfiguration to counteract operations of the eye-side variable-focuslens 540 in its respective baseline configuration. In this illustrativeexample, the baseline configuration of the real-world-side lens is +0.5diopters, and the baseline configuration of the eye-side lens is -0.5diopters, as discussed above.

To enable user 2 to utilize the HMD device 100 without needing to wearglasses or contacts, the optical power controller 710 may operate theeye-side variable-focus lens 540 in a modified configuration. Themodified configuration includes incorporating the prescription of theuser’s corrective lenses into the baseline configuration of the eye-sidevariable-focus lens. For example, if user 2 has mild myopia with acorrective lens prescription of -1.5 diopters, then the optical powercontroller 710 can control the eye-side variable-focus lens to provide-2.0 diopters of optical power, in which -1.5 diopters provides for acorrective lens prescription for user 2 and -0.5 diopters providescounteraction for the +0.5 of optical power provided by thereal-world-side variable-focus lens.

It may be appreciated that in alternative configurations, variouscombinations of optical powers can be utilized to meet particularimplementation requirements. For example, in the above scenario in whichuser 2 has a corrective lens prescription of -1.5 diopters, in the farviewing use case, the eye-side variable-focus lens 540 could becontrolled to provide -1.5 diopters of optical power and thereal-world-side variable-focus lens could be controlled to provide zerooptical power. A given lens-pair configuration can depend, for example,on physical characteristics of the HMD device and variable-focus lensessuch as switching speed/refresh rate, range of optical powers supported,display FOV, virtual image rendering plane depth, etc., as well asapplication factors such as motion blur, virtual scene composition, etc.

During close viewing by user 2, the optical power controller 710 cancontrol the eye-side variable-focus lens 540 to provide -2.0 diopters ofoptical power, as discussed above, to enable the user to simultaneouslysee both close virtual-world and real-world objects in sharp focus. Inaddition, the optical power controller adds optical power to thereal-world-side variable-focus lens 605 to push close-by real-worldobjects optically farther away and into sharp focus. The amount of addedoptical power can vary according to one or more of degree of presbyopiaexperienced by the user, level of ambient light, or other factors.

FIG. 9 is a flowchart of an illustrative method 900 for operating theHMD device 100 that includes an optical display system 500 and an eyetracker 705. Unless specifically stated, the methods or steps shown inthe flowchart and described in the accompanying text are not constrainedto a particular order or sequence. In addition, some of the methods orsteps thereof can occur or be performed concurrently and not all themethods or steps have to be performed in a given implementationdepending on the requirements of such implementation and some methods orsteps may be optionally utilized. FIG. 9 makes references to theelements shown in FIG. 7 .

At block 905, the user 115 dons the HMD device 100. Typically, the userwill have already undertaken an initialization, personalization,calibration, or other suitable processes or procedures to enhance usercomfort and/or enable, for example, various systems and subsystems toperform accurate tracking of eyes, hands, head, and/or other body parts,or provide for virtual image display alignment (e.g., if the HMD deviceshifts on the user’s head). Such processes may be utilized to determinea suitable amount of presbyopia correction to be implemented for theuser and identify the user type (e.g., user type 1 or 2 from table 800shown in FIG. 8 ). Integration of the user’s vision prescription intothe HMD device can also be supported by suchinitialization/personalization/calibration processes to improve visualcomfort and enhance mitigation effects for VAC.

At block 910, the optical power controller 710 can control the opticalpower of the eye-side variable-focus lens 540 depending on user type.The eye-side lens may be operated in its modified configurationresponsively to the user being a type 2 user. In the modifiedconfiguration, as discussed above when referring to the descriptionaccompanying FIG. 8 , appropriate optical power for the user’scorrective prescription, for example -1.5 diopters, may be added to thebaseline configuration. Otherwise, for a type 1 user, the eye-side lensoperates just in its baseline configuration to provide for the renderingof virtual images at the fixed focal plane depth (e.g., 2 m).

At block 915, the HMD device 100 including the eye-side variable-focuslens 540 is operated to render one or more virtual images at thepredetermined focal plane depth (e.g., 2 m), as appropriate for a givenHMD device user experience. At block 920, the location of the user’sgaze in the FOV of the display system is determined. The locationincludes depth along the z axis of the display and may be determined,for example, using vergence tracking of the user’s eyes, using aprojection of a gaze vector and its intersection with a rendered scene.

At decision block 925, if the determination is made from the eyetracking that the user is looking at far objects, then at block 930, theoptical power controller 710 operates the real-world-side variable-focuslens 605 in its baseline configuration. The baseline configuration ofthe real-world-side lens provides opposite optical power to that of theeye-side variable-focus lens to cancel out the impact of that lens’sbaseline configuration. For a type 1 user, this means that no netoptical power is provided to real-world image light by the variable-lenspair.

For the type 2 user, optical power is provided by the real-world-sidelens 605 to offset only the baseline optical power provided by eye-sidelens 540 without impacting the added optical power for the user’sprescription (e.g., -1.5 diopters) for the modified configuration of theeye-side lens. For example, for a 2 m virtual image focal plane, theeye-side lens is controlled to provide -0.5 diopters of optical power;therefore, the real-world-side lens is controlled to provide +0.5diopters of optical power as its baseline. This offset enables theeye-side lens to provide the prescribed correction for the type 2 user’sdistance vision.

If a determination is made at decision block 925 that the user isengaged in close viewing, then at block 935 the optical power controller710 controls the real-world-side variable-focus lens 605 to add opticalpower to push close real-world objects optically farther away and intosharp focus for the user. For example, +1.5 diopters for mild presbyopiacorrection could be added to the +0.5 diopters of baseline optical powerof the real-world-side lens.

FIG. 10 is a flowchart of an illustrative method 1000 for operating anelectronic device that includes an eye tracker and a mixed-realitysee-through optical display system for showing scenes comprising virtualimages that are rendered over views of real-world objects. At block1005, the electronic device is calibrated for utilization by apresbyopic user. Such calibration may include, for example, initiallysetting up the electronic device such as an HMD device for a particularpresbyopic user, personalizing the device to the user such as providingfor a corrective prescription, and/or performing suitable procedures toensure that various systems and subsystems in the device can accuratelyperform their functions.

At block 1010, the mixed-reality see-through optical display system isoperated to support a near field and a far field, in which the nearfield is closer to the presbyopic user relative to the far field, and inwhich the mixed-reality see-through optical display system has an eyeside and a real-world side. As noted above, 2 m may be considered athreshold between near and far fields, although other thresholddistances may be utilized depending on application requirements. Atblock 1015, a conjugate pair of variable-focus lenses are operated inmatched configurations to provide for setting rendered virtual imageswithin the near field without perturbing the views of the real-worldobjects in the far field. For example, matching configurations includethe variable-focus lenses operating to cancel the effects of theirrespective optical powers.

At block 1020, the eye tracker is used to determine a depth of thepresbyopic user’s gaze in the scene. At block 1025, responsively to adepth determination by the eye tracker, the conjugate pair ofvariable-focus lenses are operated in mismatched configurations toenable the presbyopic user to simultaneously accommodate renderedvirtual images and real-world objects in the near field. For example,the mismatch can provide for additional optical powering being added tothe real-world-side variable focus lens to thereby enable real-worldobjects in the near field to be pushed out optically and into sharpfocus by the presbyopic user.

FIGS. 11 and 12 show respective front and rear views of an illustrativeexample of a visor 1100 that incorporates an internal near-eye displaydevice 105 (FIG. 1 ) that is used in the HMD device 100 as worn by auser 115. The visor, in some implementations, may be sealed to protectthe internal display device. The visor typically interfaces with othercomponents of the HMD device such as head-mounting/retention systems andother subsystems including sensors, power management, controllers, etc.,as illustratively described in conjunction with FIGS. 16 and 17 .Suitable interface elements (not shown) including snaps, bosses, screwsand other fasteners, etc. may also be incorporated into the visor.

The visor 1100 may include see-through front and rear shields, 1105 and1110 respectively, that can be molded using transparent or partiallytransparent materials to facilitate unobstructed vision to the displaydevice and the surrounding real-world environment. Treatments may beapplied to the front and rear shields such as tinting, mirroring,anti-reflective, anti-fog, and other coatings, and various colors andfinishes may also be utilized. The front and rear shields are affixed toa chassis 1305 shown in the disassembled view in FIG. 13 .

The sealed visor 1100 can physically protect sensitive internalcomponents, including a display device 105, when the HMD device isoperated and during normal handling for cleaning and the like. Thedisplay device in this illustrative example includes left and rightwaveguides 1310 _(L) and 1310 _(R) that respectively provide holographicvirtual images to the user’s left and right eyes for mixed– and/orvirtual-reality applications. The visor can also protect the displaydevice from environmental elements and damage should the HMD device bedropped or bumped, impacted, etc.

As shown in FIG. 12 , the rear shield 1110 is configured in anergonomically suitable form 1205 to interface with the user’s nose, andnose pads and/or other comfort features can be included (e.g., molded-inand/or added-on as discrete components). In some applications, thesealed visor 1110 can also incorporate some level of optical dioptercurvature (i.e., eye prescription) within the molded shields in somecases, as discussed above. The sealed visor 1100 can also be configuredto incorporate the conjugate lens pair – the negative lens 540 andpositive lens 605 (FIG. 6 ) on either side of display device 105.

FIG. 14 shows an illustrative waveguide display 1400 having multipleDOEs that may be used as an embodiment of the see-through waveguide 510in the display device 105 (FIG. 1 ) to provide in-coupling, expansion ofthe exit pupil in two directions, and out-coupling. The waveguidedisplay 1400 may be utilized to provide holographic virtual images froma virtual imager to one of the user’s eyes. Each DOE is an opticalelement comprising a periodic structure that can modulate variousproperties of light in a periodic pattern such as the direction ofoptical axis, optical path length, and the like. The structure can beperiodic in one dimension such as one-dimensional (1D) grating and/or beperiodic in two dimensions such as two-dimensional (2D) grating.

The waveguide display 1400 includes an in-coupling DOE 1405, anout-coupling DOE 1415, and an intermediate DOE 1410 that couples lightbetween the in-coupling and out-coupling DOEs. The in-coupling DOE isconfigured to couple image light comprising one or more imaging beamsfrom a virtual image source 520 (FIG. 5 ) into a waveguide 1430. Theintermediate DOE expands the exit pupil in a first direction along afirst coordinate axis (e.g., horizontal), and the out-coupling DOEexpands the exit pupil in a second direction along a second coordinateaxis (e.g., vertical) and couples light out of the waveguide to theuser’s eye (i.e., outwards from the plane of the drawing page). Theangle ρ is a rotation angle between the periodic lines of thein-coupling DOE and the intermediate DOE as shown. As the lightpropagates in the intermediate DOE (horizontally from left to right inthe drawing), it is also diffracted (in the downward direction) to theout-coupling DOE.

While DOEs are shown in this illustrative example using a singlein-coupling DOE disposed to the left of the intermediate DOE 1410, whichis located above the out-coupling DOE, in some implementations, thein-coupling DOE may be centrally positioned within the waveguide and oneor more intermediate DOEs can be disposed laterally from the in-couplingDOE to enable light to propagate to the left and right while providingfor exit pupil expansion along the first direction. It may beappreciated that other numbers and arrangements of DOEs may be utilizedto meet the needs of a particular implementation.

As noted above, in implementations using a color model such as RGB,multiple waveguides may be utilized in the display device 105 (FIG. 1 ).FIG. 15 shows illustrative propagation of light from the virtual imagesource 520 through an optical combiner 1500 that uses a separatewaveguide for each color component in the RGB color model. Inalternative implementations, two waveguides may be utilized in which onewaveguide can support two color components and the other waveguide maysupport a single color component.

For a given angular range within the virtual FOV, light for each colorcomponent 1505, 1510, and 1515 provided by the virtual image source 520is in-coupled into respective waveguides 1530, 1535, and 1540 usingrespective individual input couplers (representatively indicated byelement 1520). The light for each color propagates through therespective waveguides in TIR and is out-coupled by respective outputcouplers (representatively indicated by element 1525) to the user’s eye115. In some implementations the output may have an expanded pupilrelative to the input in the horizontal and vertical directions, forexample when using DOEs that provide for pupil expansion, as discussedabove.

The input coupler 1520 for each waveguide 1530, 1535, and 1540 isconfigured to in-couple light within an angular range described by theFOV and within a particular wavelength range into the waveguide. Lightoutside the wavelength range passes through the waveguide. For example,the blue light 1505 is outside the range of wavelength sensitivity forboth of the input couplers in the red waveguide 1540 and green waveguide1535. The blue light therefore passes through the red and greenwaveguides to reach the in-coupling DOE in the blue waveguide 1530 whereit is in-coupled, propagated in TIR within the waveguide, propagated tothe output coupler and out-coupled to the user’s eye 115.

As noted above, the present control of variable-focus lenses in amixed-reality device for presbyopes may be utilized in mixed- orvirtual-reality applications. FIG. 16 shows one particular illustrativeexample of a mixed-reality HMD device 1600, and FIG. 17 shows afunctional block diagram of the device 1600. The HMD device 1600provides an alternative form factor to the HMD device 100 shown in FIGS.1, 2, 11, 12, and 13 . HMD device 1600 comprises one or more lenses 1602that form a part of a see-through display subsystem 1604, so that imagesmay be displayed using lenses 1602 (e.g., using projection onto lenses1602, one or more waveguide systems, such as a near-eye display system,incorporated into the lenses 1602, and/or in any other suitable manner).

HMD device 1600 further comprises one or more outward-facing imagesensors 1606 configured to acquire images of a background scene and/orphysical environment being viewed by a user and may include one or moremicrophones 1608 configured to detect sounds, such as voice commandsfrom a user. Outward-facing image sensors 1606 may include one or moredepth sensors and/or one or more two-dimensional image sensors. Inalternative arrangements, as noted above, a mixed-reality orvirtual-reality display system, instead of incorporating a see-throughdisplay subsystem, may display mixed-reality or virtual-reality imagesthrough a viewfinder mode for an outward-facing image sensor.

The HMD device 1600 may further include a gaze detection subsystem 1610configured for detecting a direction of gaze of each eye of a user or adirection or location of focus, as described above. Gaze detectionsubsystem 1610 may be configured to determine gaze directions of each ofa user’s eyes in any suitable manner. For example, in the illustrativeexample shown, a gaze detection subsystem 1610 includes one or moreglint sources 1612, such as virtual IR light or visible sources asdescribed above, that are configured to cause a glint of light toreflect from each eyeball of a user, and one or more image sensors 1614,such as inward-facing sensors, that are configured to capture an imageof each eyeball of the user. Changes in the glints from the user’seyeballs and/or a location of a user’s pupil, as determined from imagedata gathered using the image sensor(s) 1614, may be used to determine adirection of gaze.

In addition, a location at which gaze lines projected from the user’seyes intersect the external display may be used to determine an objectat which the user is gazing (e.g., a displayed virtual object and/orreal background object). Gaze detection subsystem 1610 may have anysuitable number and arrangement of light sources and image sensors. Insome implementations, the gaze detection subsystem 1610 may be omitted.

The HMD device 1600 may also include additional sensors. For example,HMD device 1600 may comprise a global positioning system (GPS) subsystem1616 to allow a location of the HMD device 1600 to be determined. Thismay help to identify real-world objects, such as buildings, etc., thatmay be located in the user’s adjoining physical environment.

The HMD device 1600 may further include one or more motion sensors 1618(e.g., inertial, multi-axis gyroscopic, or acceleration sensors) todetect movement and position/orientation/pose of a user’s head when theuser is wearing the system as part of a mixed-reality or virtual-realityHMD device. Motion data may be used, potentially along with eye-trackingglint data and outward-facing image data, for gaze detection, as well asfor image stabilization to help correct for blur in images from theoutward-facing image sensor(s) 1606. The use of motion data may allowchanges in gaze direction to be tracked even if image data fromoutward-facing image sensor(s) 1606 cannot be resolved.

In addition, motion sensors 1618, as well as microphone(s) 1608 and gazedetection subsystem 1610, also may be employed as user input devices,such that a user may interact with the HMD device 1600 via gestures ofthe eye, neck and/or head, as well as via verbal commands in some cases.It may be understood that sensors illustrated in FIGS. 16 and 17 anddescribed in the accompanying text are included for the purpose ofexample and are not intended to be limiting in any manner, as any othersuitable sensors and/or combination of sensors may be utilized to meetthe needs of a particular implementation. For example, biometric sensors(e.g., for detecting heart and respiration rates, blood pressure, brainactivity, body temperature, etc.) or environmental sensors (e.g., fordetecting temperature, humidity, elevation, UV (ultraviolet) lightlevels, etc.) may be utilized in some implementations.

The HMD device 1600 can further include a controller 1620 such as one ormore processors having a logic subsystem 1622 and a data storagesubsystem 1624 in communication with the sensors, gaze detectionsubsystem 1610, display subsystem 1604, and/or other components througha communications subsystem 1626. The communications subsystem 1626 canalso facilitate the display system being operated in conjunction withremotely located resources, such as processing, storage, power, data,and services. That is, in some implementations, an HMD device can beoperated as part of a system that can distribute resources andcapabilities among different components and subsystems.

The storage subsystem 1624 may include instructions stored thereon thatare executable by logic subsystem 1622, for example, to receive andinterpret inputs from the sensors, to identify location and movements ofa user, to identify real objects using surface reconstruction and othertechniques, and dim/fade the display based on distance to objects so asto enable the objects to be seen by the user, among other tasks.

The HMD device 1600 is configured with one or more audio transducers1628 (e.g., speakers, earphones, etc.) so that audio can be utilized aspart of a mixed-reality or virtual-reality experience. A powermanagement subsystem 1630 may include one or more batteries 1632 and/orprotection circuit modules (PCMs) and an associated charger interface1634 and/or remote power interface for supplying power to components inthe HMD device 1600.

It may be appreciated that the HMD device 1600 is described for thepurpose of example, and thus is not meant to be limiting. It may befurther understood that the display device may include additional and/oralternative sensors, cameras, microphones, input devices, outputdevices, etc. than those shown without departing from the scope of thepresent arrangement. Additionally, the physical configuration of an HMDdevice and its various sensors and subcomponents may take a variety ofdifferent forms without departing from the scope of the presentarrangement.

FIG. 18 schematically shows an illustrative example of a computingsystem that can enact one or more of the methods and processes describedabove for the present control of variable-focus lenses in amixed-reality device for presbyopes. Computing system 1800 is shown insimplified form. Computing system 1800 may take the form of one or morepersonal computers, server computers, tablet computers,home-entertainment computers, network computing devices, gaming devices,mobile computing devices, mobile communication devices (e.g.,smartphone), wearable computers, and/or other computing devices.

Computing system 1800 includes a logic processor 1802, volatile memory1804, and a non-volatile storage device 1806. Computing system 1800 mayoptionally include a display subsystem 1808, input subsystem 1810,communication subsystem 1812, and/or other components not shown in FIG.18 .

Logic processor 1802 includes one or more physical devices configured toexecute instructions. For example, the logic processor may be configuredto execute instructions that are part of one or more applications,services, programs, routines, libraries, objects, components, datastructures, or other logical constructs. Such instructions may beimplemented to perform a task, implement a data type, transform thestate of one or more components, achieve a technical effect, orotherwise arrive at a desired result.

The logic processor may include one or more processors configured toexecute software instructions. In addition, or alternatively, the logicprocessor may include one or more hardware or firmware logic processorsconfigured to execute hardware or firmware instructions. Processors ofthe logic processor may be single-core or multi-core, and theinstructions executed thereon may be configured for sequential,parallel, and/or distributed processing. Individual components of thelogic processor optionally may be distributed among two or more separatedevices, which may be remotely located and/or configured for coordinatedprocessing. Aspects of the logic processor may be virtualized andexecuted by remotely accessible, networked computing devices configuredin a cloud-computing configuration. In such a case, these virtualizedaspects may be run on different physical logic processors of variousdifferent machines.

Non-volatile storage device 1806 includes one or more physical devicesconfigured to hold instructions executable by the logic processors toimplement the methods and processes described herein. When such methodsand processes are implemented, the state of non-volatile storage device1806 may be transformed-e.g., to hold different data.

Non-volatile storage device 1806 may include physical devices that areremovable and/or built-in. Non-volatile storage device 1806 may includeoptical memory (e.g., CD, DVD, HD-DVD, Blu-Ray Disc, etc.),semiconductor memory (e.g., ROM, EPROM, EEPROM, FLASH memory, etc.),and/or magnetic memory (e.g., hard-disk drive, floppy-disk drive, tapedrive, MRAM, etc.), or other mass storage device technology.Non-volatile storage device 1806 may include nonvolatile, dynamic,static, read/write, read-only, sequential-access, location-addressable,file-addressable, and/or content-addressable devices. It will beappreciated that non-volatile storage device 1806 is configured to holdinstructions even when power is cut to the non-volatile storage device1806.

Volatile memory 1804 may include physical devices that include randomaccess memory. Volatile memory 1804 is typically utilized by logicprocessor 1802 to temporarily store information during processing ofsoftware instructions. It will be appreciated that volatile memory 1804typically does not continue to store instructions when power is cut tothe volatile memory 1804.

Aspects of logic processor 1802, volatile memory 1804, and non-volatilestorage device 1806 may be integrated together into one or morehardware-logic components. Such hardware-logic components may includefield-programmable gate arrays (FPGAs), program- andapplication-specific integrated circuits (PASIC/ASICs), program- andapplication-specific standard products (PSSP/ASSPs), system-on-a-chip(SOC), and complex programmable logic devices (CPLDs), for example.

The term “program” may be used to describe an aspect of computing system1800 typically implemented in software by a processor to perform aparticular function using portions of volatile memory, which functioninvolves transformative processing that specially configures theprocessor to perform the function. Thus, a program may be instantiatedvia logic processor 1802 executing instructions held by non-volatilestorage device 1806, using portions of volatile memory 1804. It will beunderstood that different programs may be instantiated from the sameapplication, service, code block, object, library, routine, API,function, etc. Likewise, the same program may be instantiated bydifferent applications, services, code blocks, objects, routines, APIs,functions, etc. The term “program” may encompass individual or groups ofexecutable files, data files, libraries, drivers, scripts, databaserecords, etc.

When included, display subsystem 1808 may be used to present a visualrepresentation of data held by non-volatile storage device 1806. Thisvisual representation may take the form of a graphical user interface(GUI). As the herein described methods and processes change the dataheld by the non-volatile storage device, and thus transform the state ofthe non-volatile storage device, the state of display subsystem 1808 maylikewise be transformed to visually represent changes in the underlyingdata. Display subsystem 1808 may include one or more display devicesutilizing virtually any type of technology; however, one utilizing aMEMS projector to direct laser light may be compatible with theeye-tracking system in a compact manner. Such display devices may becombined with logic processor 1802, volatile memory 1804, and/ornon-volatile storage device 1806 in a shared enclosure, or such displaydevices may be peripheral display devices.

When included, input subsystem 1810 may comprise or interface with oneor more user-input devices such as a keyboard, mouse, touch screen, orgame controller. In some embodiments, the input subsystem may compriseor interface with selected natural user input (NUI) componentry. Suchcomponentry may be integrated or peripheral, and the transduction and/orprocessing of input actions may be handled on- or off-board. Example NUIcomponentry may include a microphone for speech and/or voicerecognition; an infrared, color, stereoscopic, and/or depth camera formachine vision and/or gesture recognition; a head tracker, eye tracker,accelerometer, and/or gyroscope for motion detection and/or intentrecognition; as well as electric-field sensing componentry for assessingbrain activity.

When included, communication subsystem 1812 may be configured tocommunicatively couple various computing devices described herein witheach other, and with other devices. Communication subsystem 1812 mayinclude wired and/or wireless communication devices compatible with oneor more different communication protocols. As non-limiting examples, thecommunication subsystem may be configured for communication via awireless telephone network, or a wired or wireless local- or wide-areanetwork. In some embodiments, the communication subsystem may allowcomputing system 1800 to send and/or receive messages to and/or fromother devices via a network such as the Internet.

Various exemplary embodiments of the present control of variable-focuslenses in a mixed-reality device for presbyopes are now presented by wayof illustration and not as an exhaustive list of all embodiments. Anexample includes a mixed-reality display system that is utilizable by apresbyopic user, comprising: a see-through optical combiner throughwhich real-world objects are viewable by the user, the see-throughoptical combiner being adapted to display virtual-world images that aresuperimposed over the real-world objects over an eyebox of the displaysystem, the see-through optical combiner having an eye-side and areal-world side; a first variable-focus lens disposed on the eye-side ofthe see-through optical combiner; a second variable-focus lens disposedon the real-world side of the see-through optical combiner; and anoptical power controller operatively coupled to the first and secondvariable-focus lenses, in which the optical power controller controls abaseline configuration for each of the first and second variable-focuslenses, wherein the optical power controller is adapted to add positiveoptical power to the baseline configuration of the second variable-focuslens responsive to the presbyopic user accommodating to thepredetermined distance or less than the predetermined distance.

In another example, the baseline configuration for the firstvariable-focus lens provides negative optical power over the eyebox todisplay the virtual-world images in a focal plane at a predetermineddistance from the user, and the baseline configuration of the secondvariable-focus lens provides positive optical power to offset thenegative power of the first variable-focus lens. In another example, thebaseline configuration for the first variable-focus lens comprisesnegative optical power having of a range between -0.20 and -3.0diopters. In another example, the baseline configuration for the secondvariable-focus lens includes optical power comprising a positiveconjugate of the negative optical power of the baseline configuration ofthe first variable-focus lens. In another example, each of thevariable-focus lenses comprises technologies using one or more of liquidoil push/pull, liquid crystal, reflective MEMS (micro-electromechanicalsystem), MEMS Fresnel structures, geometric phase holograms,meta-surface optical elements, deformable membranes, Alvarez lenses, ormulti-order DOEs (diffractive optical elements). In another example, themixed-reality display system is configured for use in a head-mounteddisplay (HMD) device wearable by the presbyopic user.

A further example includes a head-mounted display (HMD) device wearableby a presbyopic user and configured for supporting a mixed-realityexperience including viewing, by the presbyopic user, of holographicimages from a virtual world that are combined with views of real-worldobjects in a physical world, comprising: a see-through display systemthrough which the presbyopic user can view the real-world objects and onwhich the holographic images are displayed within a field of view (FOV)of the see-through display system; a negative lens disposed between thesee-through display system and an eye of the presbyopic user, thenegative lens acting over the FOV and configured to render theholographic images at a focal plane having a predetermined depth fromthe presbyopic user; a variable-focus positive lens disposed on anopposite side of the see-through display system from the negative lens,the variable-focus positive lens being controllably configured to canceleffects of the negative lens on the views of the real-world objectsresponsive to the presbyopic user being engaged in viewing beyond thepredetermined depth, and the variable-focus positive lens beingcontrollably configured with increased optical power to optically pushreal-world objects into sharp focus responsive to the presbyopic userbeing engaged in viewing within the predetermined depth.

In another example, the HMD device further comprises an optical powercontroller operatively coupled to the variable-focus positive lens. Inanother example, the HMD device further comprises an eye trackeroperatively coupled to the optical power controller, the eye trackertracking vergence of the presbyopic user’s eyes or tracking a gazedirection of at least one eye of the presbyopic user, in which theoptical power controller controls the variable-focus positive lensresponsively to operations of the eye tracker. In another example, theHMD device further comprises one or more illumination sources forproducing glints for the eye tracker. In another example, the HMD devicefurther comprises one or more sensors configured to capture glints fromthe illumination sources that are reflected from features of an eye ofthe user for eye tracking. In another example, the negative lenscomprises a variable-focus lens that is operatively coupled to theoptical power controller. In another example, the optical powercontroller is configured to control the negative lens to include acorrective lens prescription for an eye of the presbyopic user. Inanother example, the corrective lens prescription provides correctionfor myopia. In another example, the see-through display system comprisesone or more waveguides that each include an input coupler and an outputcoupler, in which the input coupler is configured to in-couple one ormore optical beams for the holographic images into the waveguide from avirtual image source and the output coupler is configured to out-couplethe holographic image beams from the waveguide to an eye of thepresbyopic user, in which holographic images associated with theout-coupled beams are rendered within the FOV of the display system. Inanother example, the input coupler and output coupler each comprise adiffractive optical element (DOE) and in which each of the one or moredisplay system waveguides further comprise an intermediate DOE disposedon a light path between the input coupler and the output coupler,wherein the intermediate DOE provides exit pupil expansion of thedisplay system in a first direction and the output coupler provides exitpupil expansion of the display system in a second direction. In anotherexample, the predetermined depth is within arm’s length of thepresbyopic user.

A further example includes a method for operating an electronic devicethat includes an eye tracker and a mixed-reality see-through opticaldisplay system for showing scenes comprising virtual images that arerendered over views of real-world objects, the method comprising:calibrating the electronic device for utilization by a presbyopic user;operating the mixed-reality see-through optical display system tosupport a near field and a far field, the near field being closer to thepresbyopic user relative to the far field, and the mixed-realitysee-through optical display system having an eye side and a real-worldside; operating a conjugate pair of variable-focus lenses in matchedconfigurations to provide for setting rendered virtual images within thenear field without perturbing the views of the real-world objects in thefar field; using the eye tracker to determine a depth of the presbyopicuser’s gaze in the scene; and responsively to a depth determination bythe eye tracker, operating the conjugate pair of variable-focus lensesin mismatched configurations to enable the presbyopic user tosimultaneously accommodate rendered virtual images and real-worldobjects in the near field.

In another example, variable-focus lenses in the conjugate pair arelocated on opposite sides of the mixed-reality see-through opticaldisplay system, and in which the matched configurations comprise theconjugate pair of variable-focus lenses providing zero net optical powerto the views of the real-world objects, and in which the mismatchedconfiguration comprises optical power being added to the variable-focuslens disposed on the real-world side. In another example, the methodfurther comprises adding optical power to the variable-focus lens on theeye side to incorporate a corrective prescription of the presbyopic userfor distance vision.

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described above.Rather, the specific features and acts described above are disclosed asexample forms of implementing the claims.

The invention claimed is:
 1. A mixed-reality display system that isutilizable by a presbyopic user, comprising: a see-through opticalcombiner through which real-world objects are viewable by the user, thesee-through optical combiner being adapted to display virtual-worldimages that are superimposed over the real-world objects over an eyeboxof the display system, the see-through optical combiner having aneye-side and a real-world side; a first variable-focus lens disposed onthe eye-side of the see-through optical combiner; a secondvariable-focus lens disposed on the real-world side of the see-throughoptical combiner; an optical power controller operatively coupled to thefirst and second variable-focus lenses, in which the optical powercontroller controls a baseline configuration for each of the first andsecond variable-focus lenses; and an eye tracker operatively coupled tothe optical power controller, the eye tracker tracking vergence of thepresbyopic user’s eyes or tracking a gaze direction of at least one eyeof the presbyopic user to determine whether the presbyopic user isaccommodating to a predetermined distance, in which the optical powercontroller controls the variable-focus positive lens responsively tooperations of the eye tracker, wherein the optical power controller isadapted to add positive optical power to the baseline configuration ofthe second variable-focus lens responsive to the determination that thepresbyopic user is accommodating to the predetermined distance or lessthan the predetermined distance.
 2. The mixed-reality display system ofclaim 1 in which the baseline configuration for the first variable-focuslens provides negative optical power over the eyebox to display thevirtual-world images in a focal plane at a predetermined distance fromthe user, and the baseline configuration of the second variable-focuslens provides positive optical power to offset the negative power of thefirst variable-focus lens.
 3. The mixed-reality display system of claim2 in which the baseline configuration for the first variable-focus lenscomprises negative optical power having of a range between -0.20 and-3.0 diopters.
 4. The mixed-reality display system of claim 2 in whichthe baseline configuration for the second variable-focus lens includesoptical power comprising a positive conjugate of the negative opticalpower of the baseline configuration of the first variable-focus lens. 5.The mixed-reality display system of claim 1 in which each of thevariable-focus lenses comprises technologies using one or more of liquidoil push/pull, liquid crystal, reflective MEMS (micro-electromechanicalsystem), MEMS Fresnel structures, geometric phase holograms,meta-surface optical elements, deformable membranes, Alvarez lenses, ormulti-order DOEs (diffractive optical elements).
 6. The mixed-realitydisplay system of claim 1 as configured for use in a head-mounteddisplay (HMD) device wearable by the presbyopic user.
 7. A head-mounteddisplay (HMD) device wearable by a presbyopic user and configured forsupporting a mixed-reality experience including viewing, by thepresbyopic user, of holographic images from a virtual world that arecombined with views of real-world objects in a physical world,comprising: a see-through display system through which the presbyopicuser can view the real-world objects and on which the holographic imagesare displayed within a field of view (FOV) of the see-through displaysystem; a negative lens disposed between the see-through display systemand an eye of the presbyopic user, the negative lens acting over the FOVand configured to render the holographic images at a focal plane havinga predetermined distance from the presbyopic user; a variable-focuspositive lens disposed on an opposite side of the see-through displaysystem from the negative lens, the variable-focus positive lens beingcontrollably configured to cancel effects of the negative lens on theviews of the real-world objects responsive to the presbyopic user beingengaged in viewing beyond the predetermined distance, and thevariable-focus positive lens being controllably configured withincreased optical power to optically push real-world objects into sharpfocus responsive to a determination that the presbyopic user isaccommodating within the predetermined distance; an optical powercontroller operatively coupled to the variable-focus positive lens; andan eye tracker operatively coupled to the optical power controller, theeye tracker tracking vergence of the presbyopic user’s eyes or trackinga gaze direction of at least one eye of the presbyopic user to determinewhether the presbyopic user is accommodating within the predetermineddistance, in which the optical power controller controls thevariable-focus positive lens responsively to operations of the eyetracker.
 8. The HMD device of claim 7 further comprising one or moreillumination sources for producing glints for the eye tracker.
 9. TheHMD device of claim 8 further comprising one or more sensors configuredto captrue glints from the illumination sources that are reflected fromfeatures of an eye of the user for eye tracking.
 10. The HMD device ofclaim 7 in which the the negative lense comprises a variable-focus lensthat is operatively coupled to the optical power controller.
 11. The HMDdevice of claim 10 in which the the optical power controller isconfigured to control the negative lens to include a corrective lenseprescription for an eye of the presbyopic user.
 12. The HMD device ofclaim 11 in which the corrective lense prescription provides correctionfor myopia.
 13. The HMD device of claim 7 in which the see-throughdisplay system comprises one or more waveguides that each include aninput coupler and an output coupler, in which the input coupler isconfigured to in-couple one or more optical beams for the holographicimages into the waveguide from a virtual image source and the outputcoupler is configured to out-couple the holgoraphic image beams from thewaveguide to an eye of the presbyopic user, in which holographic imagesassociated with the out-coupled beams are rendered with the FOV of thedisplay system.
 14. The HMD device of claim 13 in which the inputcoupler and output coupler each comprise a diffractive optical element(DOE) and in which each of the one or more display sstem waveguidesfurther comprise an intermediate DOE disposed on a light path betweenthe input coupler and the output coupler, wherein the intermediate DOEprovides exit pupil exansion of the display syhstem in a first directionand the output coupler provides exit pupil expansion of the displaysystem in a secon direction.
 15. The HMD device of claim 7 in which thepredetermined depth is within arm’s length of the presbyopic user.